Learning hypernymy in distributed word vectors via a stacked LSTM network
نویسنده
چکیده
We aim to learn hypernymy present in distributed word representations using a deep LSTM neural network. We hypothesize that the semantic information of hypernymy is distributed differently across the components of the hyponym and hypernym vectors for varying examples of hypernymy. We use an LSTM cell with a replacement gate to adjust the state of the network as different examples of hypernymy are presented. We find that a seven layer LSTM model with dropout achieves a test accuracy of 81.4% on the Linked Hypernyms Dataset, though further comparison with other models in the literature is necessary to verify the robustness of these results.
منابع مشابه
Extracting keywords from email data using distributed word vectors
Current keyword extraction methods often use statistical models to select certain words from a set of documents, which fail to take advantage of the information available in the documents themselves. We propose a model for identifying semantic relationships between words in a document to identify keywords that more accurately capture the meaning of the document. Specifically, we use distributed...
متن کاملLearning Hypernymy over Word Embeddings
Word embeddings have shown promise in a range of NLP tasks; however, it is currently difficult to accurately encode categorical lexical relations in these vector spaces. We consider one such important relation – hypernymy – and investigate the feasibility of learning a function in vector space to capture it. We argue that hypernymy is significantly harder to capture than the analogy tasks word ...
متن کاملAmazon Food Review Classification using Deep Learning and Recommender System
In this paper we implemented different models to solve the review usefulness classification problem. Both feed-forward neural network and LSTM were able to beat the baseline model. Performances of the models are evaluated using 0-1 loss and F-1 scores. In general, LSTM outperformed feed-forward neural network, as we trained our own word vectors in that model, and LSTM itself was able to store m...
متن کاملSimulate Congestion Prediction in a Wireless Network Using the LSTM Deep Learning Model
Achieved wireless networks since its beginning the prevalent wide due to the increasing wireless devices represented by smart phones and laptop, and the proliferation of networks coincides with the high speed and ease of use of the Internet and enjoy the delivery of various data such as video clips and games. Here's the show the congestion problem arises and represent aim of the research is t...
متن کاملLearning Semantic Network Patterns for Hypernymy Extraction
Current approaches of hypernymy acquisition are mostly based on syntactic or surface representations and extract hypernymy relations between surface word forms and not word readings. In this paper we present a purely semantic approach for hypernymy extraction based on semantic networks (SNs). This approach employs a set of patterns sub0(a1, a2) ← premise where the premise part of a pattern is g...
متن کامل